Self-distribution binary neural networks

In this work, we study network binarization (i.e., binary neural networks, BNNs), which is one of the most promising techniques in compression for convolutional networks (CNNs). Although prior work has introduced many methods that improve accuracy BNNs by minimizing quantization error, there remains a non-negligible performance gap between binarized model and full-precision model. Given feature representation critical deep BNNs, features only differ signs, argue impact on may be strongly related to sign distribution parameters addition error. To end, Self-Distribution Binary Neural Network (SD-BNN) proposed. First, utilize Activation Self Distribution (ASD) adaptively adjust activations, thereby improving differences outputs convolution. Second, weights through Weight (WSD) then fine-tune Extensive experiments CIFAR-10 ImageNet datasets with various structures show proposed SD-BNN consistently outperforms state-of-the-art (SOTA) (e.g., 92.5% 66.5% ResNet-18) lower computational cost. Our code available at https://github.com/pingxue-hfut/SD-BNN .